Dynamically optimized transport system

ABSTRACT

A congestion control mechanism is described that is specifically designed to enhance the operation of TCP communication sessions for the delivery of web content. The congestion control mechanism dynamically adjusts the size of the congestion window in a manner that maximizes the speed of content delivery for web page requests in a cellular network. The dynamic window size adjustments, including the initial congestion control window size, are adaptive, changing as cellular network conditions change, and in a manner that is not possible with conventional TCP congestion control mechanisms that were not explicitly designed to accelerate content in cellular networks. The congestion control mechanism also learns from previous experience with a particular end user device address and network, and applies its learning to set its initial values and subsequent behavior to more optimal levels for the particular end user device and network.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority under 35 U.S.C. § 119(e) of the U.S.Provisional Patent Application Ser. No. 61/924,183, filed Jan. 6, 2014and titled, “OVERVIEW OF THE REV SOFTWARE TECHNOLOGY ANDPRODUCT—OPTIMIZING THE DELIVERY OF WEB CONTENT TO WIRELESS USERS” whichis also hereby incorporated by reference in its entirety for allpurposes.

FIELD OF THE INVENTION

The present invention relates to the field of the delivery of webcontent to wireless users.

BACKGROUND OF THE INVENTION

As the Internet has grown in scope and size over the past twenty years,there has been a corresponding exponential growth in the number of websites and web users. As these web sites have proliferated in number,size and complexity, web site performance has become an important issue.Today's web sites are considerably larger and more complex than thosethat emerged 20 years ago, which has exacerbated the performance issue.Due to the need for improved performance, technology improvements haveemerged to address this problem. For the most part, web performance hasbeen able to keep pace with user demand. However, these technologieshave focused on the wired desktop environment, which historicallyreflected the environment of most conventional users. Recently, however,the amount of web traffic to mobile users has grown to surpass that ofdesktop users.

Current approaches to speeding up the delivery of web content to endusers have been primarily focused on conventional wired networks andattempt to improve the speed of delivery by simply relocating as much ofthe web content as possible closer to the end user through the use ofContent Delivery Networks, or CDNs. With incremental improvements innetwork technology such as cloud storage and computing, content caching,and Domain Name Services (DNS), the speed of delivery has steadilyimproved. These approaches have succeeded in accelerating web contentdelivery from the web origin by caching a significant portion of webcontent at the Internet edge. By delivering web content to the user fromthe edge, rather than from the web origin site, the propagation delayhas been significantly shortened. As propagation delay may be the majorcontributor to latency, typically this means bypassing 30 to 80milliseconds of delay. For end user devices with wired connectivity, theapproach is able to reduce web content delays by 80% or more. However,for mobile end user devices such as smart phones or tablet devicesconnected to the Internet via a wireless cellular network connection,the previously described wired propagation delay may represent only afraction of the total content delay, because delay through the cellularnetwork is able to be several times the delay seen in the wired portionof the connection.

Previous approaches have not addressed the problem of content deliveryspeed over mobile cellular networks. Because cellular networks aresignificantly different in structure and design from wired networks,they present new complexities and obstacles that have not been addressedby these conventional approaches. Latencies that are generated insidethe cellular network cannot be addressed by simply moving content closerto the edge, especially when the cellular network latency is able to beseveral multiples of the propagation delay that exists from the web siteorigin to the edge.

SUMMARY OF THE INVENTION

In order to address the challenge of accelerating web and web appcontent delivery to wireless devices in a cellular network, a congestioncontrol mechanism is described that is specifically designed to enhancethe operation of TCP communication sessions for the delivery of webcontent, in both wired and wireless environments, but in particular,addresses the unique challenges of a typical cellular network. Thecongestion control mechanism dynamically adjusts the size of thecongestion window in a manner that maximizes the speed of contentdelivery for web page requests in a cellular network. The dynamic windowsize adjustments, including the initial congestion control window size,are adaptive, changing as cellular network conditions change, and in amanner that is not possible with conventional TCP congestion controlmechanisms that were not explicitly designed to accelerate content incellular networks. The congestion control mechanism also learns fromprevious experience with a particular end user device address andnetwork, and applies its learning to set its initial values andsubsequent behavior to more optimal levels for the particular end userdevice and network. The congestion control mechanism will also respondto unexpected adverse network conditions that cause packet loss, byrapidly stepping back its flow rates. By utilizing the new congestioncontrol mechanism in a proxy device at the Internet edge, web contentdelivery is significantly accelerated without the need for specializedsoftware in the end user's mobile device. Because the new congestioncontrol mechanism is perfectly compatible with the operation of previousTCP software, the pre-existing TCP/IP network software in the end userdevice is sufficient to allow the new Dynamically Optimized TransportSystem (DOTS) congestion control mechanism to operate in an “asymmetric”manner, with the web content delivery being accelerated to the end userdevice, but without any acceleration of data traffic from the end userdevice to the TCP sender managing content flow from the web page originor the Internet edge.

In an additional embodiment, having a software agent (as generated froma Software Development Kit (SDK)) on the end user device will allowfaster content delivery in the down link direction. The SDK has severalfunctions, the primary being to provide a platform for two newlydesigned data communications stacks that are based on UDP and TCP,collectively named Rev Mobile Protocol (RMP), which are specificallydesigned to optimize web content flow to and from the end user device,which is able to be a smart phone, a tablet, or a laptop with a cellularnetwork interface capability.

In one aspect, a method of executing flow control of a communicationssession over a network for acceleration of web content to mobile devicescomprises optimizing a communication for HTTP operation by allowingbursty HTTP flow characteristics and communicating data using theoptimized communication. Allowing bursty HTTP flow characteristicsincludes utilizing a bursty profile for a root object and maintainingthe bursty profile for additional objects. Optimizing the communicationfor HTTP operation includes adapting to changing network conditions in amobile network data path and a client device. The network conditionsinclude information related to wireless network type, carrier,geography, client type, latency and bandwidth. In some embodiments, themethod is implemented using cloud-based technology. Optimizing thecommunication for HTTP operation includes learning from previousexperience with a client device and the network. Optimizing thecommunication for HTTP operation includes having an adaptable initialcongestion window, slow start threshold values, adaptable congestioncontrol algorithms, selectable pacing levels, and traffic contourenhancements.

In another aspect, a method of executing congestion control of acommunications session over a network, for acceleration of web contentto mobile devices comprises optimizing a communication for HTTPoperation by allowing bursty HTTP flow characteristics, includingdynamically adjusting a size of a congestion window in a network using acongestion control mechanism and communicating data using the optimizedcommunication. Dynamically adjusting the size of the congestion windowincludes adjusting an initial congestion control window size.Dynamically adjusting the size of the congestion window changes asconditions of the network change. The congestion control mechanismlearns from previous experience with a particular end user device andthe network. In some embodiments, the method is implemented on acellular network. In some embodiments, the method is implemented on awired network.

In another aspect, a method of applying previous experience with aspecific connection in order to set appropriate parameters governing thebehavior of a TCP-type connection comprises generating and maintaining asession database of devices and networks that records specific deviceand network characteristics and performance responses in dynamicallyoptimized transport system sessions and generating and using acongestion control learning machine that uses prior session records inthe session database to set a most optimal dynamically optimizedtransport system web content delivery flow rate to a specific mobile enduser device and cellular network by analyzing the prior session recordsin order to derive the most optimal dynamically optimized transportsystem web content delivery flow rate to the specific mobile end userdevice and the cellular network by dynamically setting an initialcongestion window and setting transport rate backoff levels to be usedas a function of subsequent retransmission events. The method furthercomprises initiating a new connection with a server including obtaininga connection record, moving a record from a free container to aconnection list, using existing client records to determine a back-offlevel for a current client and calculating a client receive window andan initial congestion window. The method further comprises closing aconnection with a server including removing a connection record from aconnection list, and if a client record already exists for the clientthen the connection record is returned to a free container; otherwise,the connection record is transitioned into the client record and addedto the client list.

In another aspect, an apparatus comprises a non-transitory memory forstoring an application, the application configured for executing flowcontrol of a communications session over a network for acceleration ofweb content to mobile devices including: optimizing a communication forHTTP operation by allowing bursty HTTP flow characteristics andcommunicating data using the optimized communication and a processingcomponent coupled to the memory, the processing component configured forprocessing the application. Allowing bursty HTTP flow characteristicsincludes utilizing a bursty profile for a root object and maintainingthe bursty profile for additional objects. Optimizing the communicationfor HTTP operation includes adapting to changing network conditions in amobile network data path and a client device. The network conditionsinclude information related to wireless network type, carrier,geography, client type, latency and bandwidth. In some embodiments, thedevice is implemented using cloud-based technology. Optimizing thecommunication for HTTP operation includes learning from previousexperience with a client device and the network. Optimizing thecommunication for HTTP operation includes having an adaptable initialcongestion window, slow start threshold values, adaptable congestioncontrol algorithms, selectable pacing levels, and traffic contourenhancements.

In another aspect, an apparatus comprises a non-transitory memory forstoring an application, the application configured for executingcongestion control of a communications session over a network, foracceleration of web content to mobile devices including: optimizing acommunication for HTTP operation by allowing bursty HTTP flowcharacteristics, including dynamically adjusting a size of a congestionwindow in a network using a congestion control mechanism andcommunicating data using the optimized communication and a processingcomponent coupled to the memory, the processing component configured forprocessing the application. Dynamically adjusting the size of thecongestion window includes adjusting an initial congestion controlwindow size. Dynamically adjusting the size of the congestion windowchanges as conditions of the network change. The congestion controlmechanism learns from previous experience with a particular end userdevice and the network. In some embodiments, the apparatus isimplemented using a cellular network. In some embodiments, the apparatusis implemented using a wired network.

In yet another aspect, an apparatus comprises a non-transitory memoryfor storing an application, the application configure for applyingprevious experience with a specific connection in order to setappropriate parameters governing the behavior of a TCP-type connectionincluding: generating and maintaining a session database of devices andnetworks that records specific device and network characteristics andperformance responses in dynamically optimized transport system sessionsand generating and using a congestion control learning machine that usesprior session records in the session database to set a most optimaldynamically optimized transport system web content delivery flow rate toa specific mobile end user device and cellular network by analyzing theprior session records in order to derive the most optimal dynamicallyoptimized transport system web content delivery flow rate to thespecific mobile end user device and the cellular network by dynamicallysetting an initial congestion window and setting transport rate backofflevels to be used as a function of subsequent retransmission events anda processing component coupled to the memory, the processing componentconfigured for processing the application. The application furtherconfigured for initiating a new connection with a server includingobtaining a connection record, moving a record from a free container toa connection list, using existing client records to determine a back-offlevel for a current client and calculating a client receive window andan initial congestion window. The application further configured forclosing a connection with a server including removing a connectionrecord from a connection list, and if a client record already exists forthe client then the connection record is returned to a free container;otherwise, the connection record is transitioned into the client recordand added to the client list.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a graph of latency due to queuing delay experiencedby data in fully congested queues (buffer bloat) of various sizes.

FIG. 2 illustrates a chart of DOTS functions addressing adverse networkand device conditions.

FIG. 3 illustrates a graph of median download time through a cellularnetwork.

FIG. 4 illustrates a graph of median download time through a wirednetwork.

FIG. 5 illustrates a diagram of session information hash tablesaccording to some embodiments.

FIG. 6 illustrates a diagram of establishing connection logic accordingto some embodiments.

FIG. 7 illustrates a diagram of closing connection logic according tosome embodiments.

FIG. 8 illustrates a block diagram of an exemplary computing deviceconfigured to implement the DOTS method according to some embodiments.

FIG. 9 illustrates flowchart of a method of executing flow control of acommunications session over a network for acceleration of web content tomobile devices according to some embodiments.

FIG. 10 illustrates a flowchart of a method of executing congestioncontrol of a communications session over a network, for acceleration ofweb content to mobile devices according to some embodiments.

FIG. 11 illustrates a flowchart of a method of applying previousexperience with a specific connection in order to set appropriateparameters governing the behavior of a TCP-type connection.

FIG. 12 illustrates a diagram of a network of devices implementing theDOTS method according to some embodiments.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

With a new approach, a new congestion control mechanism is purpose-builtto address the problem of web content acceleration over cellularnetworks. The new mechanism is intended to accelerate web contentthrough a cellular network, developed specifically to overcome theobstacles presented by cellular networks. These new protocols andoptimizations are dynamic and adaptive because of the rapidly changingnetwork conditions present in the mobile network.

In order to address the problem of accelerating mobile content delivery,an entirely new approach is utilized—one that stands apart from thepresent wired network solutions. The approach utilizes an understandingof end-to-end network behavior over a combined wired and wirelessnetwork plus detailed knowledge of the inner workings of the cellularnetwork. By applying this understanding, a new data accelerationtechnology has been developed with deep roots in both the wired and thewireless worlds, resulting in a solution that provides significantperformance improvements in both realms.

Network traffic is controlled by TCP, which meters the flow of data fromone location to another through the use of several different mechanisms.An important mechanism is the TCP congestion control algorithm, whichoperates in every conventional TCP variant. These TCP variants makeassumptions about the nature of the network, and the causes of networkperformance impediments. In order to detect these impediments, the TCPvariants sense, or measure, different quantities in real time in orderto properly control the flow rate of data. There are several differentversions of the TCP congestion control algorithm, which utilize varioussources of information about network flow conditions. These sources fallinto two main classes—loss based algorithms and delay-based algorithms.Given the unique complexities of the cellular network environment, theuse of either type of algorithm will lead to non-optimal networkperformance. Many of the assumptions underlying the design of thesecongestion control algorithms are not applicable to cellular networkenvironments. As one example, the receipt of signals meant to indicatepacket loss may cause TCP to retransmit packets. However, the link layerof the cellular radio itself may trigger the re-transmission of bufferedpackets that have been lost while in transit over the air, therebyobviating the need for TCP to re-transmit.

One of the challenges to designing a TCP congestion control (CC)mechanism better suited to the cellular network environment is to ensurethat the CC mechanism does not adversely affect the pre-existing TCPflow control mechanism so that the TCP governed flow rate continues toprovide optimal content flow to the end user device by not starving theend user device or by flooding the end user device with content data.

The primary protocol used for the transfer of web content is HTTP (andHTTPS), and the system described herein specifically speeds thetransport of web content by TCP in a manner that recognizes theprotocols, mechanics and timing constraints of web page contentdownloading, and the specific characteristics of HTTP. An importantaspect of HTTP operation is the use of the SPDY protocol originallyintroduced by Google®, which provides a binary framing layer thatenables request and response multiplexing, and is a concept nowsupported by the HTTP 2.0 specification. In order to more optimallysupport efficient bandwidth utilization and improve content deliverytimes for multiplexed SPDY content, TCP's operation should be aware ofthe multiplexed HTTP streams and operate in a manner that bestaccelerates their operation.

The system described herein incorporates the intrinsic structure of atypical web page and the manner in which the individual components of atypical web page are transported by HTTP over TCP. As an example, a rootobject is usually the first structure that is transported to the enduser's web browser, and subsequent objects are transported as needed bythe root object, or by the web browser. The nature of web contenttransport, then, is best characterized as “bursty”, with the root objecttransported first, and subsequent objects transported as specified bythe root object or the user's web browser. The bursty traffic flow isable to be seen to be distinct from other types of flows such as filedownloads, video streaming, or multi-media content transport, where theflow is better characterized as continuous or smooth, rather thanbursty. Previous versions of TCP have been developed to optimize datatransport that belongs to the second group of flows, which arecontinuous in nature. The congestion control mechanisms were developedin a manner to maximize the rate of data transfer in a data channel. Bysensing delay or loss, these congestion control mechanisms act in a“fair” manner, allowing all continuous flows to share the availablebandwidth in a fair manner.

The concept of bandwidth fairness has been used as a guiding principleof design for TCP congestion control mechanisms, so that no single useris able to utilize a disproportionate amount of available bandwidth.This principle addresses the need for the equitable distribution ofnetwork bandwidth when the preponderance of TCP flows were attributableto functions such as file transfers and other applications whererelatively continuous data flows were required. With the now dominantnetwork application of web page access, data flows are much lesscontinuous, and instead are bursty in nature, because of the normaloperation of the higher layer protocol HTTP, responsible for theencoding and transfer of web content. With the bursty form of contenttransport characteristic of web content transport, TCP should operate ina manner that is not optimally supported by the prior congestion controlmechanisms that were designed for more continuous data flows. Allowingan initial burst of traffic that carries the web page root object,followed by subsequent bursts for other web page objects, permits theweb page to be downloaded and rendered by the browser more rapidly thana conventional continuous flow of content, which provides a form of“traffic shaping” to the initial burst because of the slow-start natureof conventional TCP congestion control mechanisms, which then increaseslowly over time. Because the root object is usually fairly small,conventional TCP may not have increased its flow rate substantiallyduring its slow start phase by the time the entire root object has beendelivered. The end result is that the root object, the most bottleneckedobject for subsequent web page construction, is delivered slowly.

By operating in a manner that is more responsive to the bursty flowrequirements of web content flow generated by HTTP, the system describedherein is able to significantly accelerate the delivery of web contentto the mobile end user, or even a wired desktop end user.

Challenge of Wired vs. Mobile Content Delivery

Along with the growth of the Internet and number of users, cellularnetworks have also experienced dramatic growth. Many of the improvementsto web site performance brought by technology innovations, such asContent Distribution Networks (CDN), Dynamic Site Acceleration (DSA) andHTML protocol advances have helped to mitigate the web performanceproblems experienced by users. With the use of object caching, imagecompression, and other methods to speed content delivery and reducenetwork round trips, CDNs are able to significantly reduce page loadtimes for wired (non-mobile) users. Unfortunately, the performance ofweb sites as accessed from mobile devices has always lagged behind thatof wired, desktop web access. Much of the performance improvementbrought on by these recent technology innovations has not been effectivein improving performance for mobile web users.

Limitation of Edge Caching for Mobile Content

CDNs primarily address the problem that the simple geographic distancebetween the user and the web site causes delays that become moresignificant with the increase of the number of data transfer trips tocomplete the transfer of all content to a web page. Typically, once auser accesses a web site, the contents of the web site cause manysuccessive downloads in response to HTTP GET commands sent by the enduser device or program. In part, this is due to the amount of datarequired, which necessitates numerous download trips, and due torequired interaction between the user's browser or application and theweb site itself. Each download of content is limited by the propagationdelay of the network signals being some fraction of the speed of light,and the distance from the web site to the device.

By distributing web content at the network edge, and therefore closer tothe wired end user, the most significant sources of delay have beenaddressed. Rather than addressing transport speed inefficiencies, simplyreducing the distance of the network transmission will provide areduction in content delivery time. However, for mobile users, thetransfer of content to the network edge is only a part of the overalldelay. From the network edge to the mobile user, the content traversesthe cellular network. The delay through a cellular network is usuallythe largest component of the sum of all delays experienced by thetransfer of web content from the origin web server to the end user'smobile device. For the delay through this portion of the route, CDNs areable to be of no further help, because they have already moved contentto the network edge, which is as close to the mobile device as possible.

Cellular 4G-LTE Performance Challenges

Although the technology of cellular networks has been dramaticallyimproving over the past 20 years, progressing from 2G to 2.5G, to 3G,and now with the rollout of 4G-LTE, the general latency of web access asexperienced by mobile device users has not seemed to improve in ameaningful way. While cellular network operators claim that userbandwidth has been increased in 4G-LTE, there are some suggestions thatin many cases the latency has actually increased. Many analysts haveconcluded that these cellular networks are congested, resulting in highlatencies and low bandwidth availability. Yet, there is available andunused bandwidth in most cellular networks, even while being congested,due to inefficiencies of conventional TCP and peculiarities of thecellular network as described in part in the following sections. Aproperly designed networking protocol that is able to adaptively andintelligently navigate through the network and utilize the unusedbandwidth is beneficial.

Cellular networks not only have significantly more latency, theirlatencies are also highly variable, with time constants and rates ofchange that are greater and different in behavior than wired networks.Their higher latency, lower available bandwidth, and higher variabilityare caused by a combination of factors that are intrinsic to the designand operation of cellular networks. Any attempt to provide improved webperformance through such networks involves a deep understanding of theinner characteristics of these networks. So far, the application ofexisting wired DSA and CDN solutions to mobile networks has not improvedcontent delivery performance, because they fail to address these complexsources of latency within the cellular network.

The major sources of delay or latency in cellular networks are primarilydue to reasons other than propagation delay. For cellular networks,network characteristics (bandwidth, latency, packet loss, devicediversity) are able to vary based on the identity of the cellularnetwork operator, the technology generation of the cellular networktechnology, the user service level and the device type, the operatingsystem and browser type, and the specific manner in which the mobilenetwork is provisioned. Servers and devices in the data packet pathwayof the mobile network (mobile backhaul, mobile packet core, firewalls,network address translation boxes, transparent proxies) each contributesdifferently to overall network characteristics. Therefore, in order toaddress the problem of accelerating mobile content delivery, an entirelynew approach is utilized—one that stands apart from the present wirednetwork solutions. The approach involves an understanding of end-to-endnetwork behavior over a combined wired and wireless network, and theinner workings of the cellular network. By applying this understanding,a data acceleration technology has been developed with deep roots inboth the wired and the wireless worlds, resulting in a solution thatprovides significant performance improvements in both realms.

Cellular Radio MAC Latencies

The characterization of data transfer in a mobile network as it relatesto web page performance is a multi-dimensional problem. The airinterface, the medium access control (MAC) layer, which is the radiotransmission protocol between the base station and the mobile device, isa key contributor to characteristics such as latency, packet loss, andcongestion. Because of the highly dynamic nature of radio transmission,data packets are queued in a server associated with the cellular radiotransmitter in a periodic manner, waiting for the transmission cycles ofthe radio MAC layer which controls when the packets are transmitted overthe radio medium. Radio conditions are constantly changing, and mobiledevices use time to transition their radios to active status, so datapackets may be queued for unpredictable lengths of time. When a mobileuser transfers from one cellular zone to another, a radio handoffoccurs, which may also add latency. Device queues have been designed tobe large enough so that no packets are unintentionally discarded whilewaiting for the next transmission cycle or radio handoff.

In both 3G and 4G networks, the radio connection between the user deviceand the radio base station is controlled by the Radio ResourceController, or RRC. The RRC operates in the manner of a state machine,where there are several allowed states, with transitions from one stateto another controlled by the RRC. These RRC state machine transitionsare accompanied by fixed latency intervals, which will adversely affectdata transport if not recognized and managed. The operation of the RRCimpacts latency, bandwidth, signal strength, user device battery life,and overall transmission scheduling. These in turn affect throughput,packet loss, and the efficiency and operation of conventional TCP. Anytransport acceleration methodology should recognize RRC behavior andoperate in a manner that is synergistic with RRC operation.

Cellular Devices with Small Receive Buffers

From the perspective of a TCP sender outside the cellular network, aparticular end user device may appear to have a small TCP receive buffersize, which would limit the transport rate at which content could besent by the TCP sender. This could be due either to the fact that theuser device is old and has a true hardware memory limitation, or becausethe TCP receiver in the device has been configured to send a smalladvertised window size in order to purposely slow transport rate. Ineither case, conventional TCP implementations will not be able to exceeda certain transport rate.

Bursty HTTP Traffic

In addition to the factors specific to the cellular network, HTTPtraffic in general is bursty by nature, as is the resulting responses tothe GET requests sent by end users' devices in accessing web sites. Inresponse to the first HTTP GET command, the web server transmits theinitial root object as a burst of data. Because receipt of the initialroot object is a prerequisite to the end user browser or app for furtherprocessing, third (3^(rd)) party link fetching and page rendering, it isadvantageous to deliver that object as quickly as possible so that theseprocesses are able to start. Any traffic shaping or buffering of theroot object will only delay the processing and rendering of the webpage. Therefore, DOTS has been designed to maintain the original burstyprofile of the root object transport. Likewise, all following objectsare transported in bursts, rather than being traffic-shaped into flatterflows.

Buffer Bloat

A significant contributor to latency is the use of large memory buffersin servers and devices in the data path. Paradoxically, the use of largepacket buffers has a detrimental effect on the performance of TCP,leading to degraded performance because of increased queuing delay, asshown in FIG. 1. When large buffers remain full, now described as“buffer bloat”, TCP will be adversely affected leading to largelatencies. The intent of device and network designers in using liberalamounts of memory was to prevent packet loss in overloaded networks.Standard TCP congestion control algorithms keep buffers full atbottleneck links, and the congestion control algorithms do not operatewell in these environments because notification of packet drops (theindicator of congestion for these protocols), are delayed due to thelarge amount of buffering that Occurs.

The existence of these large buffers has become an additional obstaclefor the efficient transport of web content to mobile users.

Most of the widely deployed TCP implementations use loss-basedcongestion control, where the TCP sender will not slow down its sendingrate until the TCP sender senses packet loss. The original design ofTCP, and the dozen or so conventional variants of TCP are all based onthe assumption that either packet loss or delay is able to be used as ameasure of congestion with the receiver sending ACKs to acknowledgereceipt of packets. With the existence of very large buffers in thecellular network, TCP senders do not have available to them the usualpacket loss or delay signals that are required for proper TCP operation.In the environment, the TCP sender is not properly sensing congestion,and sends packets at an incorrect or non-optimal rate, with the resultthat the full bandwidth available for data transport is not utilizedefficiently.

Because packet loss is either prevented by the large buffers, or packetloss is concealed through the operation of link-layer retransmissions inthe cellular network, the TCP sender receives fewer indications ofpacket loss and will continue to increase its sending rate. This willresult in nearly filling most of the buffer space, largely increasingthe total RTT, or Round Trip Time experienced by the TCP flow, andthereby dramatically reducing the efficiency of data flow.

Many smart phone manufacturers have compensated for this buffer bloatfactor by setting a small value for the smart phone's “advertised TCPmaximum receive buffer” size, even though the actual physical buffersize in the phone may be much larger. The advertised receive windowcannot be larger than the receive buffer size, and the TCP sender cannotsend more data than allowed by the advertised receive window, so the TCPcongestion window is limited. However, this will limit the throughput tothe device and unnecessarily degrade the transmission rate. TCP is againnot able to operate at the maximum efficiency possible. A new TCPimplementation designed to optimize and accelerate content flow througha cellular network should address these factors.

Performance Enhancing Proxies

Some cellular network operators have inserted performance enhancingproxies (PEPs) into their networks as a means to control and trafficshape bursty incoming flows, or to act as transparent caches. These PEPsterminate the TCP sessions of origin senders (websites) and buffertraffic for retransmission towards the end user device. These PEPs aretransparent, in that the senders cannot distinguish the PEP from the enduser device and accept and acknowledge all data regardless of data rateor burstiness, masking the end user device characteristics from the TCPsender. The masking effect is able to limit the efficiency ofconventional TCP and the visibility of the end user devicecharacteristics. The masking effect also adds complexity to the designand operation challenges of an adaptive and optimized TCP. The PEPs areable to adversely affect the transport rate of content if they invoketraffic shaping and flow control of the incoming content stream.

Network Bandwidth Asymmetry

Another common characteristic of cellular networks is the asymmetricbandwidth distribution of the uplink versus the downlink directions.Conventional TCP congestion control algorithms provide throughput thatis not only a function of the link and the traffic characteristics inthe direction of data transfer. Studies have demonstrated that link andtraffic characteristics in the reverse direction have a significanteffect on forward direction throughput. This is because existing TCPimplementations are ACK-clocked, and the total bandwidth available inthe reverse-direction is able to be significantly lower than in theforward direction. This results in forward-direction web content trafficsuffering significant degradation, with the link capacity beingsignificantly underutilized. This then is a major contributor toinefficient data flow through cellular networks because TCP is notoperating to efficiently utilize all available bandwidth. There may besignificant bandwidth capacity in the cellular network that is not beingutilized, but could be with the proper transport technology.

Mobile RestAPis

The increasingly prevalent role of mobile web apps should be considered.Web apps access web sites through an access technology called RestAPls,in addition to the conventional HTML access used by desktopapplications. Some mobile apps embed HTML access within a software“shell”, representing yet another type of access. Each type of access isaccompanied with a particular set of performance constraints that shouldbe considered in designing an acceleration technology for mobile users.

Dynamic Network Conditions

Cellular network conditions are dynamic and subject to rapid changes.Therefore, designing a simple static solution, such as a conventionalCDN that is optimal for all cellular networks at all times is notpossible. A dynamic, adaptable solution is utilized—one that quantifiespresent network conditions and end user device environments, and adaptsaccordingly in order to maintain the most optimal performance possibleeven while conditions change. This represents the singular challenge indeveloping a working solution for the acceleration of web content tomobile users.

DOTS

The primary focus of the DOTS software system is to provide adaptiveend-to-end acceleration of web content to mobile users. The technologythat provides the acceleration does so in a manner that is aware of enduser apps, adapts to changing conditions in the mobile network data pathand the client device, and takes action to optimize content delivery byadaptively managing data streams. Coincidentally, the DOTS Softwaresystem also provides adaptive end-to-end acceleration to non-mobiledevices with the use of the same technology. Because the technology isadaptive, it senses the different conditions and environment of a wirednetwork also, and similarly accelerates transport.

In some embodiments, the system adaptively optimizes every end usersession from end-to-end without requiring any additional software to beinstalled on the end user device. This configuration accelerates thetransport of web content to the mobile device, but does not acceleratedata traffic in the reverse direction. In another configuration of thesystem, RMP, transport of content and data is accelerated symmetrically,meaning both from the origin (web server) towards the mobile device, andfrom the mobile device towards the origin. However, this functionalityinvolves the mobile device hosting one of several versions of clientsoftware in the form of a software SDK library that a third-party app iscompiled with, or a software client application.

The rapid delivery of data to mobile devices is extremely challenging,where latency in the so-called last mile is able to range from 50milliseconds to several seconds or more, depending on network conditionsand has wide variability. The latency is additive to the latency thatexists in the first mile and the middle mile of the transport sequence.

The system also provides content optimization and edge caching in amanner similar to traditional CDNs, but with the intent of providing aplatform that not only decreases the required number of round tripsthrough the wired network, but also serves as a sensing and datagathering platform for the real time tuning and adaptation for itsmobile network transport acceleration function and content optimizationfunctions. With the characteristics of mobile networks changing veryrapidly on a real time basis, a fixed network transport solution isunlikely to be appropriate even seconds after that particular solutionperformed well. Consequently, the software utilizes real time sensing ofnetwork conditions with the use of Real User Measurements (RUM) in orderto provide data necessary to generate real time fine tuning of transportbehavior for these rapidly changing network conditions.

The DOTS Software collects the data from the combined wired and cellularnetwork segments to identify and tune approaches for accelerated contenttransport and web content optimization strategies. Data is gathered fromreal world networks, both wired and cellular, in real time, and fromnetwork emulation sessions in the software internal network.

First, the overall network behavior is characterized which includes thetargeted collection of network performance data, as the network existsin its present state. The system should efficiently, accurately, and ina repeatable fashion, collect raw network and client device performancedata that exists in four separate categories:

By wireless network type: Examples include LTE, 3G, GPRS, WiFi.By carrier: Examples include Comcast, AT&T, Verizon, T-Mobile, Sprint.By geography: Examples include cities in the U.S., Europe, Asia.By client type: Examples include device operating system, browser type,PC, Smartphone, tablet.

The DOTS Software system adapts to the characteristics of the cellularnetwork environment in real time, so the most optimal performancestrategies are able to be used.

The system has a data transmission technology that compensates for theadverse effects of buffer bloat commonly seen in cellular networks, andthe asymmetric bandwidth link pathways of cellular networks along withseveral other unique characteristics of the cellular network. With theuse of the technology, the system is able to take advantage of otherwiseunused bandwidth in the cellular network and accelerate datatransmission without negatively impacting other data.

Web performance has become more critical than ever, primarily due to thepersonalization of content, making the content non-cacheable, and alsothe vastly growing amount of mobile access. Current optimizationtechniques are static and one-size-fits-all. Optimization tools alsotend to be layer specific—presentation layer (Front End Optimizer, orFEO), transport layer (CDN, TCP Optimizers).

DOTS System Architecture

DOTS exists in a larger system (the Software System), which is cloudbased, with Points of Presence (POPs) distributed in many domestic andinternational locations. The overall architecture is similar to that ofa conventional CDN, except that the Software System targets mobile usersand has a last-mile transport acceleration component. The CDN typearchitecture is one that provides the infrastructure for the operationof the DOTS system. The overall Software System is described here toprovide context for the DOTS component.

In some embodiments, all software servers (or dedicated or specifiedservers) are connected to each other with redundant, high speed datapathways in a segment of a software cloud referred to as the “middlemile.” The POPs are positioned such that they are geographically closeto customer origin sites, end user locations, or both. Software serversthat are in POPs close to the customer origin server, function as“content optimizers.” The portion of the network pathway from thecustomer origin to the software content optimizer is called the “firstmile.” The content optimizer servers act to analyze web traffic that issent by the customer origin server in response to an end user browser'srequest. The content is then optimized for more rapid and efficienttransport to the end user through the execution of front endoptimizations such as image compression, re-sequencing of web pageobjects, Java Script and CSS optimization.

After in-line processing the requested content is then forwarded by thecontent optimizer to a software “browser proxy” that is geographicallysituated in close proximity to the end user. The primary purpose of thebrowser proxy is to collect and forward all content to the end userdevice, while at the same time providing a single, unified pathway forall traffic to and from the end user device. The browser proxy functionsto consolidate and coordinate all content, forwarding the content viaDOTS to the end user device in the most expeditious manner possible,rather than having content flow through various pathways to the end userdevice, without coordination or acceleration.

The pathway between the content optimizer and the browser proxy istermed the “middle mile.” After receipt of the content, the browserproxy determines whether any content is cacheable, and generates cacheentries for those objects that are found to be cacheable. Additionally,any third party content is recognized, and requests for that content aresent from the browser proxy to the corresponding third parties. Contentin response to these third party requests is sent back through thebrowser proxy (where the content is also potentially cached for futureuse) and delivered via DOTS' accelerated transport protocol, along withother non-3rd party content, to the end user device.

With the appropriate amount of content, the browser proxy forwards thecontent to the end user device in what is termed the “last mile” portionof the network.

The Software System has a policy engine and network transportoptimization software that in some embodiments combines real-timesession data with the ability to apply optimizations holistically inreal time and on a session-specific basis—dynamically at the start ofthe session and adaptively during the session. Such in-sessionadjustments can be important for mobile access as latency and congestionare able to vary dramatically over time, and the ability to reactappropriately to these changing conditions is important for acceleratingtransport and maintaining an acceptable user experience.

The Software System is an adaptive, policy based content and networkoptimizer that facilitates better web user experience and networkbandwidth utilization from the provider perspective. The policies,generated and managed by the policy engine, are based on the content,network and end-user device awareness. The policies are not onlyconfigurable, but also updated dynamically based on the performance ofthe Software System at the session granularity level, making the changesactionable to facilitate dynamic optimization. A distributedarchitecture model is used to facilitate scaling, optimal compute andnetwork resource utilization, isolation in a shared environment andreliability. In addition, the system is able to handle security attacksin a graceful manner.

Web content acceleration performance in general has become more complexand difficult due to the increasing personalization of content, makingmuch of the content non-cacheable. Current content optimizationtechniques are generally static and one-size-fits-all. Optimizationtools also tend to be layer specific—presentation layer (FEO, or frontend optimization), server response (back end response) and transportlayer (CDN, TCP Optimizers). The software approach is also organized inlayers. One layer provides in-line content optimization of content thatincludes such functions as image compression, rewriting Java script toremove white space, combine java scripts, rewrite and combine CSS files,combine CSS elements, lazyload images.

A policy engine manages the combination of real-time session data withthe ability to apply optimizations holistically and on asession-specific basis; dynamically at the start of the session andadaptively during the session. In-session adjustments are especiallyimportant for mobile access as latency and congestion is able to changea lot, and the ability to react to these changing conditions isimportant for maintaining an acceptable user experience.

Providing performance data to quantify the user experience is animportant component of the Software System. In addition to providingperformance data to the customer, RUM (Real User Measurements) data isused by the policy controller to adjust and adapt session configurationand content optimization strategies to changing network and usecharacteristics. As customer content passes through the contentoptimizer servers, small, tight Java Scripts are “injected” into thecontent that provide timing information that is collected and analyzedby the system while serving the web pages to the end user. TheJavascript files execute in the end user's browser and collectinformation from the browser reflecting the timing definitions of theW3C's NavTiming API. The page load timings and network characteristicsare sent to a web service, the RUM Aggregator, which accumulates data ina database for offline analysis. The specific data collected includesnetwork performance, network latency, web site response times, page loadtimes, and other related data. Some of the data is used for optimizationwithin the content optimizer. The data is also used to compute the ROIand report generation for customers and for internal consumption. Withthe collection of the data, the Policy Manager is able to adapt thecontent optimization and transport mechanisms to the changing conditionstypical of mobile environments. The proactive tester is the frameworkthat provides the RUM synthetic testing mechanism.

DOTS Transport Technology

Even when web content is able to be delivered from a contentdistribution network's edge server, typically within 10 millisecondsfrom a “wired” end-user, the mobile “last mile” is able to add 50milliseconds to several seconds of latency. Even a latency of 50milliseconds adds the equivalent delay of that seen in sending contentfrom LA to NY. Therefore even with a content distribution network thatmoves content to the network edge, the total content delivery timethrough a cellular network is able to remain very high. Therefore, a newgeneration of transport technology is able to bridge the gap of themobile network.

Two specialized data transport protocols and congestion controlmechanism/algorithms have been developed to accelerate the delivery ofweb content in the last mile for both mobile and wired networks. Oneprotocol, DOTS, is TCP based, and transports content in a unidirectionalmanner from the Software System to the end user and does not require anyadditional software in the end user device. The other protocol, RMP, isboth UDP and TCP based and transports content bi-directionally betweenthe system and the end user; the protocol utilizes additional softwarein the end user device.

DOTS is a specialized TCP compatible protocol that has been purposelybuilt to accelerate web content in a unidirectional manner over the lastmile segment of the network. DOTS is designed to be adaptable to thechanging conditions present in cellular networks while also acceleratingtransport of web content over wired networks which typically have lowerrates of change than cellular networks. The transport protocol,utilizing the latest advances in modern network queuing theory andcommunications protocol design, is designed to be compatible with theoperation of conventional TCP implementations. Because of this, DOTSresides in a server node near the network edge and transports contentover the last mile, either mobile or wired, with that content receivedby a conventional TCP instance in the end user device.

By applying any previous experience with the particular end user IPaddress, prior network traffic conditions, cellular network type,cellular operator identity, device type, DOTS learns, adapts andmodifies its configuration and operation to maximize the throughput tothe mobile or wired end user device. DOTS will configure itselfappropriately for each transmission on a session by session basis, so ifnetwork conditions have changed, DOTS will respond and act accordingly.DOTS will accelerate web content to the maximum rate consistent withpresent network conditions and end user device capabilities. With thenext access by the end user, DOTS will apply the configuration it haslearned is best for that end user device and network as a starting pointso that the transport acceleration DOTS provides has already benefittedfrom any prior experience. While transporting content, DOTS continuallyadjusts itself to changing conditions on the fly, maximizing itsperformance while sensing any limiting conditions that utilize real timemodification of DOTS behavior. DOTS will then store relevant sessiondata relating to connection conditions and make that informationavailable for any subsequent DOTS session by that end user.

DOTS will adapt to real time conditions and implement a specificbehavior and optimize its configuration by having an adaptable initialcongestion window, slow start threshold values, adaptable congestioncontrol algorithms, selectable pacing levels, and traffic contourenhancements. Since DOTS is able to adapt and configure itself, DOTS isable to maintain a time-averaged behavior that is consistent withconventional TCP network expectations of traffic fairness.

DOTS operates in a manner that allows unused bandwidth to be utilizedfor web content delivery. In cellular networks, a common networkcondition is bandwidth asymmetry between the downlink and uplinkdirections, which slows transport rate of conventional TCPimplementations. This condition is sometimes caused by buffer bloat,which itself is a major contributor to underutilized bandwidth capacityin the network. Instead of utilizing ACK-clocking to regulate its dataflow rates as do conventional TCP implementations, DOTS utilizes apacing algorithm called RBE (Bandwidth Estimator) that is not adverselyaffected by either asymmetrical bandwidth link characteristics or bufferbloat. The approach has proven to provide a significant improvement inbandwidth utilization in asymmetric link environments and buffer bloatconditions common in cellular networks and some wired networks. DOTSalso utilizes leaky bucket flow management algorithms to control thepacing of packet transmissions in the absence of reliable RTT data.

Transport technology is focused on the optimization and acceleration ofHTTP web traffic. Typically, an HTTP GET command, issued by the enduser's browser, results in an HTTP response from the server that firstincludes the root object. Only after receipt of the root object will theuser's browser be able to begin executing and rendering the requestedweb site content; this represents a bottleneck or critical path item.DOTS is designed to maintain the burst character of the root objectresponse so that the end user device receives the root object in thefastest time possible. DOTS invokes an initial flow burst that is notmoderated by a conventional TCP slow start algorithm, instead drivingthe initial object at the maximum rate that is able to be supported bythe network and the end user device. The duration of the initial burstphase is controlled by a leaky bucket algorithm, so that once the rootobject is transferred, DOTS enters a modified congestion avoidance modecontrolling the transport rate with input from RBE as described above.

With the existence of a Performance Enhancing Proxy (PEP) in the datapath, conventional TCP algorithms are unable to generate an end to endconnection between the website server and the end user device, therebybeing subject to any traffic shaping or flow control behavior imposed bythe proxy. In the presence of PEP's traffic shaping and flow controleffect, both DOTS and RMP transport technology have the ability tosometimes bypass these PEP devices and thereby establish a higher flowrate to the end user device without being subject to PEP-mediatedtraffic shaping or flow control.

While the common condition of buffer bloat in cellular networks slowscontent flow managed by conventional TCP implementations, DOTS is notmaterially affected by buffer bloat, and content will still beaccelerated.

DOTS overcomes problems compounded by the small value for the smartphone's TCP maximum receive buffer size, the asymmetric link pathways ofcellular networks, along with several other unique characteristics ofthe cellular network. With the use of the technology described herein,software is able to take advantage of the unused bandwidth in thecellular network, and accelerate data transmission without negativelyimpacting other data.

With the addition of the SDK and the RMP, the system will monitor theend user device and the radio state, including battery status, signalstrength, cellular network type, and the Radio Resource Controller statemachine. Using the radio data, the system will be able to predictchanging MAC latencies and optimize content flow more efficiently.

Because of the highly dynamic environment of cellular networks, DOTS hasthe ability to adapt to these changing conditions in real time. DOTSalso stores relevant session data concerning network and devicecharacteristics, which allows DOTS to apply that experience to the samenetwork environment by initializing the start of the new session withparameters of the most optimal values as retrieved from the priorsession record. FIG. 2 shows DOTS functions addressing adverse networkand device conditions.

Transport Technology Performance

While the transport technology is specifically designed to acceleratecontent transport through the cellular network, and indeed achievessignificant content acceleration through the cellular network, thetransport technology also provides significant acceleration throughnon-cellular, wired networks where the last mile may include DSL, DOCSIScable, or fiber optics.

FIGS. 3 and 4 chart the relative performance of DOTS vs. conventionalTCP. FIG. 3 graphs the download time of data objects of various sizesthrough a typical cellular network. FIG. 4 graphs the download time ofdata objects of various sizes through a typical wired network. Similargains are seen through the networks of other carriers, such as AT&T.

DOTS Description

A DOTS version of the TCP congestion control mechanism is described. TCPDOTS utilizes a combination of methods to manage the congestion window(CWND) that are specific to cellular networks, and optimizedspecifically for the rapid transport of HTTP messages. Throughout thisdescription, examples of metrics that are able to be used to determinethe initial CWND are described. It should be apparent to those skilledin the art that more or less metrics are able to be used to make thisdetermination of the initial CWND.

In all that follows, the focus is on cellular networks which havedistinctly different network characteristics than wired, being moredynamic and variable than wired (the original target of TCP design), andtherefore utilizes a more dynamic approach to the conventional TCPmechanisms such as initial and current congestion window sizing, slowstart algorithm.

DOTS is based on targeted modifications of the Westwood TCP code. Themain functionality of Westwood and DOTS is to estimate the bandwidth andcalculate how many packets are able to be sent out on the wire given thecurrent conditions. There is a subtle difference between the formulaeused by each:

Westwood: #pkts=(BWe*RTTmin)/MSS

DOTS: #pkts=(BWe*RTT)/MSS

Westwood is conservative in its calculation and uses the smallest RTTever seen on the connection. DOTS on the other hand uses the last RTTmeasurement and is able to therefore determine that the connection isable to handle more packets. At no point will the DOTS calculation beless than the Westwood calculation.

The calculation comes into play when a connection experiences congestionor packet loss. In both cases, Westwood uses the formula above to setthe slow start threshold, ssthresh. The threshold is used to reset thecongestion window, CWND, once it has recovered from the congestion eventor packet loss. In the case of congestion, DOTS sets ssthresh towhichever is greater, the value calculated using the formula above or90% of the current CWND. In the case of packet loss, DOTS sets thessthresh to whichever is greater, the value calculated using the formulaor 75% of the current CWND.

When the connection is in the congestion avoidance state (steady state)and out of the slow start phase, Westwood increases the CWND at the samerate as that used in TCP Reno, a TCP congestion-avoidance algorithm. TheCWND is incremented by 1 when a window's worth of ACKs is received. Forexample, if the CWND is 20, the connection should receive 20 ACKs beforethe CWND is increased to 21. DOTS uses the same algorithm used by Cubicto grow the CWND. The algorithm allows DOTS to increase the CWND moreaggressively while the current value is less than the calculatedmaximum. A difference between the DOTS version of the algorithm andCubic's algorithm is that Cubic updates a delayed ACK variable everytime an ACK is received. This controls how fast or slow the CWND isincreased. The DOTS version of the algorithm does not update the delayedAck value.

DOTS Initial Congestion Window

The initial congestion window used by TCP dictates how many packets areable to be transmitted to a client in the initial burst of data.Standard Linux TCP currently uses 10. This, with a standard maximumsegment size (MSS) of 1460 limits the initial burst of data to a clientto 14,600 bytes. On top of this, the congestion window growth is slow,as standard TCP utilizes a slow start algorithm where the congestionwindow is increased by 1 for every ACK received. All of these combinedresults in sub-optimal initial bandwidth utilization on data objects >14KB. As an example, given a data object of 64 KB, the complete objectwill be transmitted to the client long before standard TCP grows thecongestion window enough to make full use of the available bandwidth.

DOTS does not use a fixed initial congestion window. Each client isdifferent. They run on variety of operating systems, browsers, networkswith different network conditions. So a single, fixed initial congestionwindow does not provide optimum bandwidth for all clients. Therefore,DOTS uses the client data to determine what the initial congestionwindow should be. First, DOTS looks at the advertised receive window inthe client's ACK packet of the three-way TCP handshake and determines ifthe window is a small (<64 KB), medium (>=64 KB & <128 KB) or large(>=128 KB) window. The classification determines the multiplier used toset a new receive window for client: 3 for small, 2 for medium and 1 forlarge. Secondly, DOTS divides the new receiver window by the agreed uponMSS. For a window size of 16K, the initial congestion window will be16,384*3/1460=33. A congestion window of this size will allow the serverto transmit data objects up to 47 KB in size during the initial burst.While the use of three window classifications is described here, thesystem described herein includes the possibility of using more thanthree classifications.

There are times when a client advertises a receiver window that it trulycannot handle. In this scenario, a larger initial congestion windowcould actually cause congestion or packet loss. If this does occur, thesession database is updated with the appropriate information so that theinitial congestion window is not so aggressive in the future for thegiven client.

DOTS Learning Machine—Session Database

The DOTS Session database is used by DOTS to maintain metrics about allactive and past DOTS connections. The data is used to determine how wellDOTS is working, which clients have experienced problems with the DOTSand to what degree.

The database is comprised of two types of records. First is the sessionrecord. Every active TCP connection to the server has a session record,which the client's IP address and port number uniquely identify. Therecord is used to record the latency and bandwidth measurements ascalculated by the DOTS. Second is the client record, which is a summaryof past connections from a particular IP address. DOTS maintains ahistory of the total packets sent, total retransmissions, bandwidth(low, high and last), latency (low, high and last) and back-off level.These values are maintained for three TCP receiver window sizes: small(less than 64 KB), medium (between 64 KB and 128 KB) and large (greaterthan 128 KB). While the use of three TCP window size values is usedhere, the system described herein includes the possibility ofmaintaining more than three values.

The generation of session and client records occurs within the DOTSstack interrupt handler, which means no blocking operations arepermitted. Therefore, the memory for new records should bepre-allocated. In order to not consume more memory than necessary,records are allocated in blocks of 300. As these records are utilized,it is possible that the number of available “free” records will dropbelow an acceptable threshold (currently one third of the original blocksize). If and when this occurs, a new block will be allocated. In someembodiments, there is no limit on the number of blocks that are able tobe allocated. The reason being that session records are returned to the“free” pool once the connection is closed and client records are limitedto the number of unique IP addresses that have connected to the server.If these reasons fail to hold true in the future, an algorithm will beutilized to reclaim old client records and ensure the system memory isnot exhausted.

The DOTS session database provides APIs to generate new records, updatemeasurement data within session records, and retrieve client data fromwithin the kernel and from the application layer. The database does notprovide APIs to generate or edit client records as this is managedcompletely within the database.

DOTS Flow Rate Control and Congestion Control

DOTS is able to be considered somewhat aggressive with regards to datatransfer rates. DOTS does attempt to maximize the transfer rate for allclients but at a rate that is proportional to the client's advertisedreceive window. However, not every client is able to successfully handlethe higher transfer rate. This is able to result in failed transfers,connection resets, excessive packet retransmission, and longer transfertimes. The congestion control learning machine functionality is a meansto help prevent these issues on future connections from a particularclient (IP address). Congestion control learning machine analyzesvarious TCP metrics as a connection is closing and determines whether ornot future connections should back off in the level of aggressiveness.When a new connection is established, DOTS will query the sessiondatabase for the client's back-off level. If the connection has neverexperienced any issues then there is no back-off level. Otherwise aback-off level is given based on the severity of the issues seen on pastconnections. The back-off level indicates the extent to which DOTS isable to manipulate the client's advertised receive window (ARW) and theinitial congestion window (ICW).

In some embodiments, the congestion control learning machine tracks howmany packets were transmitted for a connection and how many packets areto be retransmitted and uses the data to determine the back-off level.In order to give the appropriate weighting to connections thatexperienced issues, any connection that successfully transmits all ofits data without a single retransmission is not used to determine theback-off level. The congestion control learning machine calculates theretransmission percentage and uses that as the back-off level. Forexample, if a connection experiences less than 10% retransmission, itsback-off level is 0. If a connection experiences between 10% and 19%retransmission rate, then its back-off level is 1 and so on.

As previously mentioned, DOTS manipulates the ARW and ICW. DOTS appliesa multiplier to the ARW based upon its original value. If the ARW isless than 64 KB, then it is multiplied by 3. If the ARW is between 64 KBand 128 KB, the multiplier is 2. For any ARW larger than 128 KB, themultiplier is 1. The ICW is determined by dividing the new ARW by theMSS (normally 1448). If necessary, the ICW is capped at 60. With thecongestion control learning machine, DOTS will alter its manipulationbased on the back-off level. For any back-off level other than 0, theARW multiplier will be 1, and the calculated ICW is divided by theback-off level and has a minimum threshold of 10. For example, if theARW is 14 KB, and the back-off level is 0, then the new ARW is 42 Kb,and the ICW is 29. If the ARW is 14 KB and the back-off level is 2 thenthe new ARW is 14 KB and ICW is 14.

Implementation

The DOTS Session Database should be lightweight, fast and not a CPU hog.In order to accomplish these goals, the foundation for the database isable to be either a double linked list or a hash table. In Linux, thehash table and standard list implementations are very similar but from aspeed point of view, the hash table is slightly better. The standardlist is a double linked list whereas the hash table is a single linkedlist, meaning one less pointer update on every addition/deletion fromthe list. For the purpose of the session table, this is able to be alarge CPU savings, as records are added and deleted from the table asoften as connections are opened and closed. Another factor affectingspeed is how quickly searches for specific clients are able to becompleted. Extremely fast search capability could be accomplished if asingle layer hash table is used, and the client's IP address is used asthe hash key. This would result in a quick index into the hash table tolook up any client requiring no hash function or data comparison.However, this would include the ability to store one client record forevery possible IP address or 4,294,967,296 records plus additionalrecords for any active connection (session). The memory usage for theimplementation is significant, especially when most of the memory wouldnever be used: 588,514,304 records never used as they correspond toreserved IP addresses, and it is unlikely that every other IP addresswould initiate a connection to each and every server (or a specified setof servers). A two layer hash table and hash function are used to reducethe memory usage and still provide quick searching. In some embodiments,the hash function is the hash_32( ) function in standard Linux, and thehash_32( ) function will be used in conjunction with the two mostsignificant octets in the client's IP address. This reduces the numberof records at startup to 65536 records. As more and more clients connectto the server, this number will grow, but it will get nowhere near thenumber of records in the single layered solution. This will also allowthe session database to group client information according to theirservice provider.

FIG. 5 illustrates the connection and client hash tables as maintainedwithin the database. A session information hash table 500 includes aconnection list 502 and a client list 504. The connection list 502includes client hash keys, and for each client hash key, there is anassociated client IP address and port number. The client list 504includes client hash keys with an associated client IP address. Anotherhash table is used within the session database. It is a single layerhash table that does not use a hash key to quickly reference entries.The table is used to maintain a list of all allocated records that arenot currently in use. The table could be maintained as a list instead ofa hash table, but in order to simplify the moving of records from onetable to the next, a hash is used.

FIG. 6 illustrates the processing logic that is executed when a clientinitiates a new connection with the server. FIG. 6 shows how a newconnection obtains a connection record, moving a record from the FreeContainer to the Connection List, how the existing client records areused to get the back-off level for the current client and how the newARW and ICW are calculated.

In the step 600, a connection is initiated by a client. In the step 602,the DOTS CCA Init is executed which gets a connection record 604 fromthe Free Record Container 606. In the step 608, the new connectionrecord is added to the Connection List 610. In the step 612, the DOTSPost SYN Configuration function (POST SYN Config) is executed whichrequests the back-off level for the current client from the Client List614. In the step 616, it is determined if the back-off level is equal to0. If it is determined that the back-off is not equal to 0, then in thestep 618, the advertised receive window (ARW) is left unchanged and theinitial congestion window (ICW) is calculated by dividing the ARW by thecurrent maximum segment size (MSS) and then dividing the result by theback-off level (ICW=ARW/MSS/Backoff Level). If it is determined that theback-off level is equal to 0, then in the step 620, the ARW ismultiplied by the determined multiplier and the ICW is calculated bydividing the new ARW by the current MSS (ARW*=Multiplier & ICW=ARW/MSS).In the step 622, the connection is marked as Established.

FIG. 7 illustrates the processing executed when a client-initiatedconnection is being closed. FIG. 7 shows how the connection record isremoved from the Connection List, how the connection metrics are updatedin the corresponding client record (new record created if necessary) andhow the connection record is returned to the Free Record Container.

In the step 700, a connection close is initiated. In the step 702, theConnection Record 704 is removed from the Connection List 706. In thestep 708, the connection status is updated and a Client Record 710 isretrieved from the existing Client List 712. In the step 714, it isdetermined if the retrieval of the Client Record returned a validrecord. If it is determined that the Client Record did not return avalid record, then at the step 716, a new Client Record 718 is retrievedfrom the Free Record Container 724. If it is determined that the ClientRecord did return a valid record, or after the new Client Record 718 isretrieved at the step 716, then at the step 720, the client record isupdated with the current connection's statistics. The connection recordis returned to the Free Record Container 724 in the step 722. In thestep 726, the connection is closed.

FIG. 8 illustrates a block diagram of an exemplary computing deviceconfigured to implement the DOTS system according to some embodiments.The computing device 800 is able to be used to acquire, store, compute,process, communicate and/or display information. In general, a hardwarestructure suitable for implementing the computing device 800 includes anetwork interface 802, a memory 804, a processor 806, I/O device(s) 808,a bus 810 and a storage device 812. The choice of processor is notcritical as long as a suitable processor with sufficient speed ischosen. The memory 804 is able to be any conventional computer memoryknown in the art. The storage device 812 is able to include a harddrive, CDROM, CDRW, DVD, DVDRW, Blu-Ray®, flash memory card or any otherstorage device. The computing device 800 is able to include one or morenetwork interfaces 802. An example of a network interface includes anetwork card connected to an Ethernet or other type of LAN. The I/Odevice(s) 808 are able to include one or more of the following:keyboard, mouse, monitor, display, printer, modem, touchscreen, buttoninterface and other devices. In some embodiments, the hardware structureincludes multiple processors and other hardware to perform parallelprocessing. DOTS system application(s) 830 used to perform the DOTSmethod are likely to be stored in the storage device 812 and memory 804and processed as applications are typically processed. More or fewercomponents shown in FIG. 8 are able to be included in the computingdevice 800. In some embodiments, DOTS hardware 820 is included. Althoughthe computing device 800 in FIG. 8 includes applications 830 andhardware 820 for implementing the DOTS method, the DOTS method is ableto be implemented on a computing device in hardware, firmware, softwareor any combination thereof. For example, in some embodiments, the DOTSapplications 830 are programmed in a memory and executed using aprocessor. In another example, in some embodiments, the DOTS hardware820 is programmed hardware logic including gates specifically designedto implement the method. In some embodiments, the computing device isalso able to be a virtual machine (VM) architecture.

In some embodiments, the DOTS application(s) 830 include severalapplications and/or modules. In some embodiments, modules include one ormore sub-modules as well.

Examples of suitable computing devices for implementing the DOTS and/orthe RMP as described herein, include a personal computer, a laptopcomputer, a computer workstation, a server, a mainframe computer, ahandheld computer, a personal digital assistant, a cellular/mobiletelephone (e.g. an iPhone®), a smart appliance, a smart television, atablet computer (e.g. an iPad®), a smart watch, networking devices(e.g., a proxy device, hub, router, switch), a gaming device or anyother suitable computing device.

FIG. 9 illustrates flowchart of a method of executing flow control of acommunications session over a network for acceleration of web content tomobile devices according to some embodiments. In the step 900, acommunication for HTTP operation is optimized. The communication is ableto be optimized in any manner such as by allowing bursty HTTP flowcharacteristics. For example, burst flow behavior is utilized for rootobject delivery and secondary object delivery. In the step 902, data iscommunicated using the optimized communication. In some embodiments,fewer or additional steps are implemented. In some embodiments, theorder of the steps is modified.

FIG. 10 illustrates a flowchart of a method of executing congestioncontrol of a communications session over a network, for acceleration ofweb content to mobile devices according to some embodiments. In the step1000, a communication for HTTP operation is optimized by allowing burstyHTTP flow characteristics, including dynamically adjusting a size of acongestion window in a network using a congestion control mechanism. Inthe step 1002, data is communicated using the optimized communication.In some embodiments, fewer or additional steps are implemented. In someembodiments, the order of the steps is modified.

FIG. 11 illustrates a flowchart of a method of applying previousexperience with a specific connection in order to set appropriateparameters governing the behavior of a TCP-type connection. In the step1100, a session database of devices and networks that records specificdevice and network characteristics and performance responses indynamically optimized transport system sessions is generated andmaintained. In the step 1102, a congestion control learning machine thatuses prior session records in the session database is generated and usedto set a most optimal dynamically optimized transport system web contentdelivery flow rate to a specific mobile end user device and cellularnetwork by analyzing the prior session records in order to derive themost optimal dynamically optimized transport system web content deliveryflow rate to the specific mobile end user device and the cellularnetwork by dynamically setting an initial congestion window and settingtransport rate backoff levels to be used as a function of subsequentretransmission events. In some embodiments, fewer or additional stepsare implemented. In some embodiments, the order of the steps ismodified.

FIG. 12 illustrates a diagram of a network of devices implementing theDOTS method according to some embodiments. A network of devices 1200includes a server 1202 (or other computing device such as a proxydevice), a network 1204 and one or more networked devices (e.g., a smartphone 1206, a tablet 1208, a personal computer/laptop 1210, and/or asmart television 1212). As described, the server 1202 is able tocommunicate with the one or more networked devices over the network 1204in an optimized manner by implementing the DOTS method. The network 1204is able to be any type of network such as a wired, wireless, a LocalArea Network (LAN), a larger network, the Internet, a cellular networkand/or any other network or type of network, and/or a combinationthereof.

To utilize the DOTS method and system, an end user utilizes a device(e.g., a smart phone) as the user typically would; however, the userwould be able to perform communications via a network in a much fasterand more optimized manner.

In operation, the DOTS method and system accelerates the transport ofdata on a network. For example, a user browsing the web using a smartphone would receive web page content much more quickly by communicatingusing the DOTS method and system.

The present invention has been described in terms of specificembodiments incorporating details to facilitate the understanding ofprinciples of construction and operation of the invention. Suchreference herein to specific embodiments and details thereof is notintended to limit the scope of the claims appended hereto. It will bereadily apparent to one skilled in the art that other variousmodifications may be made in the embodiment chosen for illustrationwithout departing from the spirit and scope of the invention as definedby the claims.

1. A method of applying previous experience with a specific connectionto set appropriate parameters governing the behavior of a TCP-typeconnection comprising: a. generating and maintaining a session databaseof devices and networks that records specific device and networkcharacteristics and performance responses in dynamically optimizedtransport system sessions; and b. initiating a new connection with aserver including obtaining a connection record, moving a record from afree container to a connection list, using existing client records todetermine a back-off level for a current client and calculating a clientreceive window and an initial congestion window.
 2. (canceled)
 3. Themethod of claim 1 further comprising closing a connection with a serverincluding removing a connection record from a connection list, and if aclient record already exists for the client then the connection recordis returned to a free container; otherwise, the connection record istransitioned into the client record and added to the client list.
 4. Anapparatus comprising: a. a non-transitory memory for storing anapplication, the application configured for applying previous experiencewith a specific connection to set appropriate parameters governing thebehavior of a TCP-type connection including: i. generating andmaintaining a session database of devices and networks that recordsspecific device and network characteristics and performance responses indynamically optimized transport system sessions; and ii. initiating anew connection with a server including obtaining a connection record,moving a record from a free container to a connection list, usingexisting client records to determine a back-off level for a currentclient and calculating a client receive window and an initial congestionwindow; and b. a processing component coupled to the memory, theprocessing component configured for processing the application. 5.(canceled)
 6. The apparatus of claim 4 wherein the application isfurther configured for closing a connection with a server includingremoving a connection record from a connection list, and if a clientrecord already exists for the client then the connection record isreturned to a free container; otherwise, the connection record istransitioned into the client record and added to the client list.
 7. Amethod of applying previous experience with a specific connection to setappropriate parameters governing the behavior of a TCP-type connectioncomprising: a. generating and maintaining a session database of devicesand networks that records specific device and network characteristicsand performance responses in dynamically optimized transport systemsessions; and b. closing a connection with a server including removing aconnection record from a connection list, and if a client record alreadyexists for the client then the connection record is returned to a freecontainer; otherwise, the connection record is transitioned into theclient record and added to the client list.
 8. The method of claim 7further comprising initiating a new connection with a server includingobtaining a connection record, moving a record from a free container toa connection list, using existing client records to determine a back-offlevel for a current client and calculating a client receive window andan initial congestion window.
 9. An apparatus comprising: a. anon-transitory memory for storing an application, the applicationconfigured for applying previous experience with a specific connectionto set appropriate parameters governing the behavior of a TCP-typeconnection including: i. generating and maintaining a session databaseof devices and networks that records specific device and networkcharacteristics and performance responses in dynamically optimizedtransport system sessions; and ii. closing a connection with a serverincluding removing a connection record from a connection list, and if aclient record already exists for the client then the connection recordis returned to a free container; otherwise, the connection record istransitioned into the client record and added to the client list; and b.a processing component coupled to the memory, the processing componentconfigured for processing the application.
 10. The apparatus of claim 9wherein the application is further configured for initiating a newconnection with a server including obtaining a connection record, movinga record from a free container to a connection list, using existingclient records to determine a back-off level for a current client andcalculating a client receive window and an initial congestion window.